A review of algorithms and software for real-time electric field modeling techniques for transcranial magnetic stimulation.

Transcranial magnetic stimulation (TMS) is a device-based neuromodulation technique increasingly used to treat brain diseases. Electric field (E-field) modeling is an important technique in several TMS clinical applications, including the precision stimulation of brain targets with accurate stimulation density for the treatment of mental disorders and the localization of brain function areas for neurosurgical planning. Classical methods for E-field modeling usually take a long computation time. Fast algorithms are usually developed with significantly lower spatial resolutions that reduce the prediction accuracy and limit their usage in real-time or near real-time TMS applications. This review paper discusses several modern algorithms for real-time or near real-time TMS E-field modeling and their advantages and limitations. The reviewed methods include techniques such as basis representation techniques and deep neural-network-based methods. This paper also provides a review of software tools that can integrate E-field modeling with navigated TMS, including a recent software for real-time navigated E-field mapping based on deep neural-network models.

SlicerTMS: Real-Time Visualization of Transcranial Magnetic Stimulation for Mental Health Treatment.

We present a real-time visualization system for Transcranial Magnetic Stimulation (TMS), a non-invasive neuromodulation technique for treating various brain disorders and mental health diseases. Our solution targets the current challenges of slow and labor-intensive practices in treatment planning. Integrating Deep Learning (DL), our system rapidly predicts electric field (E-field) distributions in 0.2 seconds for precise and effective brain stimulation. The core advancement lies in our tool's real-time neuronavigation visualization capabilities, which support clinicians in making more informed decisions quickly and effectively. We assess our system's performance through three studies: First, a real-world use case scenario in a clinical setting, providing concrete feedback on applicability and usability in a practical environment. Second, a comparative analysis with another TMS tool focusing on computational efficiency across various hardware platforms. Lastly, we conducted an expert user study to measure usability and influence in optimizing TMS treatment planning. The system is openly available for community use and further development on GitHub: https://github.com/lorifranke/SlicerTMS.

N-Tools-Browser: Web-Based Visualization of Electrocorticography Data for Epilepsy Surgery.

Epilepsy affects more than three million people in the United States. In approximately one-third of this population, anti-seizure medications do not control seizures. Many patients pursue surgical treatment that can include a procedure involving the implantation of electrodes for intracranial monitoring of seizure activity. For these cases, accurate mapping of the implanted electrodes on a patient's brain is crucial in planning the ultimate surgical treatment. Traditionally, electrode mapping results are presented in static figures that do not allow for dynamic interactions and visualizations. In collaboration with a clinical research team at a Level 4 Epilepsy Center, we developed N-Tools-Browser, a web-based software using WebGL and the X-Toolkit (XTK), to help clinicians interactively visualize the location and functional properties of implanted intracranial electrodes in 3D. Our software allows the user to visualize the seizure focus location accurately and simultaneously display functional characteristics (e.g., results from electrical stimulation mapping). Different visualization modes enable the analysis of multiple electrode groups or individual anatomical locations. We deployed a prototype of N-Tools-Browser for our collaborators at the New York University Grossman School of Medicine Comprehensive Epilepsy Center. Then, we evaluated its usefulness with domain experts on clinical cases.

How Machine Learning is Powering Neuroimaging to Improve Brain Health.

This report presents an overview of how machine learning is rapidly advancing clinical translational imaging in ways that will aid in the early detection, prediction, and treatment of diseases that threaten brain health. Towards this goal, we aresharing the information presented at a symposium, "Neuroimaging Indicators of Brain Structure and Function - Closing the Gap Between Research and Clinical Application", co-hosted by the McCance Center for Brain Health at Mass General Hospital and the MIT HST Neuroimaging Training Program on February 12, 2021. The symposium focused on the potential for machine learning approaches, applied to increasingly large-scale neuroimaging datasets, to transform healthcare delivery and change the trajectory of brain health by addressing brain care earlier in the lifespan. While not exhaustive, this overview uniquely addresses many of the technical challenges from image formation, to analysis and visualization, to synthesis and incorporation into the clinical workflow. Some of the ethical challenges inherent to this work are also explored, as are some of the regulatory requirements for implementation. We seek to educate, motivate, and inspire graduate students, postdoctoral fellows, and early career investigators to contribute to a future where neuroimaging meaningfully contributes to the maintenance of brain health.

Two Stream Active Query Suggestion for Active Learning in Connectomics.

For large-scale vision tasks in biomedical images, the labeled data is often limited to train effective deep models. Active learning is a common solution, where a query suggestion method selects representative unlabeled samples for annotation, and the new labels are used to improve the base model. However, most query suggestion models optimize their learnable parameters only on the limited labeled data and consequently become less effective for the more challenging unlabeled data. To tackle this, we propose a two-stream active query suggestion approach. In addition to the supervised feature extractor, we introduce an unsupervised one optimized on all raw images to capture diverse image features, which can later be improved by fine-tuning on new labels. As a use case, we build an end-to-end active learning framework with our query suggestion method for 3D synapse detection and mitochondria segmentation in connectomics. With the framework, we curate, to our best knowledge, the largest connectomics dataset with dense synapses and mitochondria annotation. On this new dataset, our method outperforms previous state-of-the-art methods by 3.1% for synapse and 3.8% for mitochondria in terms of region-of-interest proposal accuracy. We also apply our method to image classification, where it outperforms previous approaches on CIFAR-10 under the same limited annotation budget. The project page is https://zudi-lin.github.io/projects/#two_stream_active.

TRAKO: Efficient Transmission of Tractography Data for Visualization.

Fiber tracking produces large tractography datasets that are tens of gigabytes in size consisting of millions of streamlines. Such vast amounts of data require formats that allow for efficient storage, transfer, and visualization. We present TRAKO, a new data format based on the Graphics Layer Transmission Format (glTF) that enables immediate graphical and hardware-accelerated processing. We integrate a state-of-the-art compression technique for vertices, streamlines, and attached scalar and property data. We then compare TRAKO to existing tractography storage methods and provide a detailed evaluation on eight datasets. TRAKO can achieve data reductions of over 28x without loss of statistical significance when used to replicate analysis from previously published studies.

Peax: Interactive Visual Pattern Search in Sequential Data Using Unsupervised Deep Representation Learning.

We present Peax, a novel feature-based technique for interactive visual pattern search in sequential data, like time series or data mapped to a genome sequence. Visually searching for patterns by similarity is often challenging because of the large search space, the visual complexity of patterns, and the user's perception of similarity. For example, in genomics, researchers try to link patterns in multivariate sequential data to cellular or pathogenic processes, but a lack of ground truth and high variance makes automatic pattern detection unreliable. We have developed a convolutional autoencoder for unsupervised representation learning of regions in sequential data that can capture more visual details of complex patterns compared to existing similarity measures. Using this learned representation as features of the sequential data, our accompanying visual query system enables interactive feedback-driven adjustments of the pattern search to adapt to the users' perceived similarity. Using an active learning sampling strategy, Peax collects user-generated binary relevance feedback. This feedback is used to train a model for binary classification, to ultimately find other regions that exhibit patterns similar to the search target. We demonstrate Peax's features through a case study in genomics and report on a user study with eight domain experts to assess the usability and usefulness of Peax. Moreover, we evaluate the effectiveness of the learned feature representation for visual similarity search in two additional user studies. We find that our models retrieve significantly more similar patterns than other commonly used techniques.

Evaluating 'Graphical Perception' with CNNs.

Convolutional neural networks can successfully perform many computer vision tasks on images. For visualization, how do CNNs perform when applied to graphical perception tasks? We investigate this question by reproducing Cleveland and McGill's seminal 1984 experiments, which measured human perception efficiency of different visual encodings and defined elementary perceptual tasks for visualization. We measure the graphical perceptual capabilities of four network architectures on five different visualization tasks and compare to existing and new human performance baselines. While under limited circumstances CNNs are able to meet or outperform human task performance, we find that CNNs are not currently a good model for human graphical perception. We present the results of these experiments to foster the understanding of how CNNs succeed and fail when applied to data visualizations.

NeuroBlocks--Visual Tracking of Segmentation and Proofreading for Large Connectomics Projects.

In the field of connectomics, neuroscientists acquire electron microscopy volumes at nanometer resolution in order to reconstruct a detailed wiring diagram of the neurons in the brain. The resulting image volumes, which often are hundreds of terabytes in size, need to be segmented to identify cell boundaries, synapses, and important cell organelles. However, the segmentation process of a single volume is very complex, time-intensive, and usually performed using a diverse set of tools and many users. To tackle the associated challenges, this paper presents NeuroBlocks, which is a novel visualization system for tracking the state, progress, and evolution of very large volumetric segmentation data in neuroscience. NeuroBlocks is a multi-user web-based application that seamlessly integrates the diverse set of tools that neuroscientists currently use for manual and semi-automatic segmentation, proofreading, visualization, and analysis. NeuroBlocks is the first system that integrates this heterogeneous tool set, providing crucial support for the management, provenance, accountability, and auditing of large-scale segmentations. We describe the design of NeuroBlocks, starting with an analysis of the domain-specific tasks, their inherent challenges, and our subsequent task abstraction and visual representation. We demonstrate the utility of our design based on two case studies that focus on different user roles and their respective requirements for performing and tracking the progress of segmentation and proofreading in a large real-world connectomics project.

Altered Structural Brain Networks in Tuberous Sclerosis Complex.

Tuberous sclerosis complex (TSC) is characterized by benign hamartomas in multiple organs including the brain and its clinical phenotypes may be associated with abnormal neural connections. We aimed to provide the first detailed findings on disrupted structural brain networks in TSC patients. Structural whole-brain connectivity maps were constructed using structural and diffusion MRI in 20 TSC (age range: 3-24 years) and 20 typically developing (TD; 3-23 years) subjects. We assessed global (short- and long-association and interhemispheric fibers) and regional white matter connectivity, and performed graph theoretical analysis using gyral pattern- and atlas-based node parcellations. Significantly higher mean diffusivity (MD) was shown in TSC patients than in TD controls throughout the whole brain and positively correlated with tuber load severity. A significant increase in MD was mainly influenced by an increase in radial diffusivity. Furthermore, interhemispheric connectivity was particularly reduced in TSC, which leads to increased network segregation within hemispheres. TSC patients with developmental delay (DD) showed significantly higher MD than those without DD primarily in intrahemispheric connections. Our analysis allows non-biased determination of differential white matter involvement, which may provide better measures of "lesion load" and lead to a better understanding of disease mechanisms.

Design and Evaluation of Interactive Proofreading Tools for Connectomics.

Proofreading refers to the manual correction of automatic segmentations of image data. In connectomics, electron microscopy data is acquired at nanometer-scale resolution and results in very large image volumes of brain tissue that require fully automatic segmentation algorithms to identify cell boundaries. However, these algorithms require hundreds of corrections per cubic micron of tissue. Even though this task is time consuming, it is fairly easy for humans to perform corrections through splitting, merging, and adjusting segments during proofreading. In this paper we present the design and implementation of Mojo, a fully-featured single-user desktop application for proofreading, and Dojo, a multi-user web-based application for collaborative proofreading. We evaluate the accuracy and speed of Mojo, Dojo, and Raveler, a proofreading tool from Janelia Farm, through a quantitative user study. We designed a between-subjects experiment and asked non-experts to proofread neurons in a publicly available connectomics dataset. Our results show a significant improvement of corrections using web-based Dojo, when given the same amount of time. In addition, all participants using Dojo reported better usability. We discuss our findings and provide an analysis of requirements for designing visual proofreading software.

Regional infant brain development: an MRI-based morphometric analysis in 3 to 13 month olds.

Elucidation of infant brain development is a critically important goal given the enduring impact of these early processes on various domains including later cognition and language. Although infants' whole-brain growth rates have long been available, regional growth rates have not been reported systematically. Accordingly, relatively less is known about the dynamics and organization of typically developing infant brains. Here we report global and regional volumetric growth of cerebrum, cerebellum, and brainstem with gender dimorphism, in 33 cross-sectional scans, over 3 to 13 months, using T1-weighted 3-dimensional spoiled gradient echo images and detailed semi-automated brain segmentation. Except for the midbrain and lateral ventricles, all absolute volumes of brain regions showed significant growth, with 6 different patterns of volumetric change. When normalized to the whole brain, the regional increase was characterized by 5 differential patterns. The putamen, cerebellar hemispheres, and total cerebellum were the only regions that showed positive growth in the normalized brain. Our results show region-specific patterns of volumetric change and contribute to the systematic understanding of infant brain development. This study greatly expands our knowledge of normal development and in future may provide a basis for identifying early deviation above and beyond normative variation that might signal higher risk for neurological disorders.